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[NAME OF DOCUMENT] SCOPE OF CLAIMS 
[CLAIM 1] 

An audio reproduction method when reproducing audio data to which 
character data for displaying a character having a specific shape is added, 
5 comprising: 

generating data on an image having the shape specified by said character 
data by analyzing the character data, and 

displaying the generated image data correspondingly to the reproduction 
of said audio data. 
10 [CLAIM 2] 

An audio reproduction method according to claim 1, 
wherein when motion data for indicating motion of the character having the shape 
specified by said character data is further added to said audio data, the motion 
indicated by the motion data is performed correspondingly to the reproduction of 
15 said audio data. 

[CLAIM 3] 

An audio reproduction method according to claim 1, 
wherein said character data is data on a character having a three-dimensional 
shape, and based on a predetermined input operation the character to be displayed 
2 0 is made into a character having a shape seen from an arbitrary viewpoint. 

[CLAIM 4] 

An audio reproduction apparatus comprising: 

retaining means for retaining audio data to which character data for 
displaying a character having a specific shape is added, 

2 5 audio reproducing means for processing to reproduce the audio data 

retained in said retaining means, 

image processing means for generating data on an image having the 
shape specified by the character data by analyzing the character when character 
data is added to the audio data reproduced in said audio reproducing means, and 

3 0 display means for displaying image data generated in said image 

processing means correspondingly to the reproduction in said audio reproducing 

means. 

[CLAIM 5] 

An audio reproduction apparatus according to claim 4, 

3 5 wherein motion data for indicating motion of the character having the shape 

specified by said character data is further added to the audio data retained in said 
retaining means, and 

said image processing means generates an image in which the motion 
indicated by said motion data is added to the character specified by said character 

4 0 data. 

[CLAIM 6] 

An audio reproduction apparatus according to claim 4, 
wherein the character data added to the audio data retained in said retaining 
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means is data on a character having a three-dimensional shape, 

operating means for indicating a viewpoint toward the character having a 

three-dimensional shape is provided, and 

based on the viewpoint indicated by said operating means, the image 
5 data generated in said image processing means is made into an image of a 

character seen from the viewpoint. 
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[NAME OF DOCUMENT] SPECIFICATION 

[TITLE OF THE INVENTION] AUDIO REPRODUCTION METHOD AND 
APPARATUS 
[TECHNICAL FIELD] 
5 [0001] 

The present invention relates to an audio reproduction method and 
apparatus for reproducing audio data stored in some medium or audio data 
downloaded, for example. 
[BACKGROUND ART] 
10 [0002] 

A conventional audio reproduction apparatus such as a stereo 
reproduction apparatus is designed for processing to reproduce audio data 
recorded in a recording medium such as an installed CD (compact disc) and MD 
(mini disc), or audio data received from the outside. On this occasion, there are 
15 cases where the following visual display processing is performed at the time of 

reproduction in a reproduction apparatus, that is, for example, a display panel is 
provided as a spectrum analyzer for displaying a level variation and the like, of 
every bandwidth analyzed in the spectrum analyzer, in music under reproduction. 
[0003] 

2 0 Patent Document 1 discloses the display of a spectrum analyzer in audio 

equipment. 

[Patent Document 1] Japanese Patent Application No. H8- 130425 
[DISCLOSURE OF THE INVENTION] 
[PROBLEMS TO BE SOLVED BY THE INVENTION] 

2 5 [0004] 

Hereupon, the display according to a spectrum analyzer and the like in 
the past only indicates data characteristics of reproduced music and the like, and 
so there is a problem that information associated with data on the music is not 
displayed positively. In other words, it is difficult for the display in 

3 0 conventional reproduction apparatus of this type to display motion associated 

with the music under reproduction or to display a character of singer of the music. 
[0005] 

In order to solve this problem, it is conceived, for example, to prepare a 
medium capable of recording image data such as DVD (Digital Video Disc or 

3 5 Digital Versatile Disc), in which the image data is recorded together with audio 

data, and to display the image based on that image data at the time of reproducing 
audio data, however, such image data has a large data volume, and if such image 
data is handled in a typical audio reproduction apparatus, a heavy burden will be 
imposed thereon, which raises another problem. 

4 0 [0006] 

The present invention has been made in view of these points and aims at 
enabling the image attached to the audio to be displayed comparatively with ease. 
[MEANS FOR SOLVING THE PROBLEMS] 
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[0007] 

The present invention is an audio reproduction method when 
reproducing audio data to which character data for displaying a character having a 
specific shape is added, the method including: generating data on an image 
5 having the shape specified by the character data by analyzing the character data, 

and displaying the generated image data correspondingly to the reproduction of 
said audio data. 

[0008] 

By doing in this way, correspondingly to the audio reproduction, the 
1 0 character by the character data added to the audio data will be displayed. 

[EFFECT OF THE INVENTION] 
[0009] 

According to the present invention, at the time of audio reproduction, the 
character corresponding to the audio reproduction will be displayed. Thus, as 
15 compared to the case of separately preparing dynamic data, it is possible to 

perform the display corresponding to the audio with smaller data size. 

[0010] 

In this case, when motion data for indicating motion of the character 
having the shape specified by said character data is further added, the motion 

2 0 indicated by the motion data is performed correspondingly to the reproduction of 

said audio data. By doing in this way, the character's motion linked with the 
audio reproduction can be displayed and, for example, choreography 
corresponding to music and the like can be known from the motion of the 
displayed character. 
25 [0011] 

The character data are data showing a character of a three-dimensional 
shape and based on a predetermined input operation the character to be displayed 
is made into a character having a shape seen from an arbitrary viewpoint. By 
doing so, the character seen from an arbitrary direction can be displayed based on 

3 0 the operation of the user, and the character can be displayed in the form 

preferable for the user. 

[BEST MODE FOR CARRYING OUT THE INVENTION] 
[0012] 

Hereinafter, an embodiment according to the present invention will be 

3 5 described with reference to accompanying drawings. 

FIG. 1 shows an example of a configuration of an audio reproduction 
apparatus in this embodiment. In case of this example, it is designed that a 
recording medium 11 for recording audio data is installed in the reproduction 
apparatus. The recording medium 11 may be, for example, an optical disc or 

4 0 magneto-optical disc such as a CD and MD in which digital audio data is 

recorded, or various kinds of memory card. In addition, a semiconductor 
memory, hard disc and the like incorporated in the reproduction apparatus may be 
employed in which audio data downloaded from the outside is recorded. In this 
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example, it is arranged that character data formed of data on a shape expressing 
some object is added to the audio data recorded in the recording medium 11. A 
specific example of the character data will be described later. 
[0013] 

5 The audio data recorded in the recording medium 1 1 is read out by a data 

reader 12. When there is data such as the character data added to the audio data, 
the added data is also read out by the data reader 12 at the same time. The 
readout data is supplied to a data processor 13, where data processing such as 
error correction is performed, after that, the audio data is supplied to an audio 

10 reproduction processor 14, where audio reproduction processing is performed. 

The reproduction audio data processed in the audio reproduction processor 14 is 
supplied to a digital/analog converter 15 and is converted into analog audio 
signals of a right channel and left channel, and a converted analog audio signal of 
each channel is amplified by amplifiers 16L, 16R and then is supplied to speakers 

15 17L, 17R for respective channels to be output. 

[0014] 

The character data read out together with audio data by the data reader 
12 is separated from the audio data by the data processor 13 and supplied to a 
character-data processor 21. The character-data processor 21 determines 
2 0 contents of the character data to generate data for displaying an image having the 

shape specified by the character data, and the generated image data is supplied to 
an image processor 22 and is made into image data of a predetermined format, 
which is supplied to a display panel 23 to be displayed thereon. When a display 
device is, for example, incorporated in the reproduction apparatus, a liquid crystal 

2 5 display panel or the like can be applied as the display panel 23. 

[0015] 

Audio processing by the audio processor 14, character processing by the 
character-data processor 21 and the like are performed under the control of a 
controller 24 which is a central control unit. A memory 25 storing a control 

3 0 program and the like is connected to the controller 24. Further, it is configured 

that the controller 24 receives instructions by operation of an operating key 26. 
In case of this example, possible operation by the operating key 26 includes 
operation for determining a viewpoint of the displayed character as well as 
operation related to the reproduction of audio. 

3 5 [0016] 

When the audio reproduction apparatus in this example configured as 
described above is seen from the processing of character data, a hierarchic 
structure as shown in FIG. 2 is conceivable. Specifically, the whole of the audio 
data reproduction apparatus 2 is controlled by OS (Operating System) 1 installed 

4 0 in the controller 24; when the audio reproduction is performed by the 

reproduction apparatus 2, if character data is added to audio data, processing on 
the character data is performed by a character engine 3 including the data 
processor 13, character-data processor 21, image processor 22, display panel 23 
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and the like. The character engine 3 includes a construction analysis module 3a, 
a performance module 3b, and a display module 3c. 
[0017] 

FIG. 3 is a flowchart showing the flow of data processing in audio 
5 reproduction apparatus of this example. First, when audio data is read out from 

the recording medium 11 (step Sll), the controller 24 judges whether or not the 
character data and motion data are added to the audio data read (step S12). 
When it is judged that the character data and motion data are not added, only the 
audio data read out from the recording medium 11 (step SI 3) is extracted to 

10 perform reproduction processing to be output from the speakers 17L and 17R 

(step S14). When it is judged that the character data and motion data are added 
in step SI 2, processing of separating audio data from the other data (character 
data and motion data) is performed (step SI 5) and reproduction processing is 
performed with respect to the separated audio data in the following step S14. 

15 [0018] 

Then, the character data and motion data separated in step S15 are 
subjected to construction analysis (step SI 6), and image processing for generating 
an image based on the analyzed construction is performed (step SI 7) and the 
generated image is displayed on the display panel (step SI 8). 

2 0 [0019] 

FIG. 4 shows an example of the processing state: when audio data 100 
including the character data and motion data is processed to be reproduced, the 
character data is separated from the audio data in a section functioning as the 
construction analysis module 3 a, and the performance module 3b and display 

2 5 module 3c perform processing on the data judged as the character data to be 

displayed as the character. With respect to the character data and motion data, 
the construction analysis module 3 a performs processing of changing the internal 
data structure into an easy-to-handle form for the performance module. The 
processed character data is expressed as a connection between a joint and regions 

3 0 in a manner that corresponds to the structure of human body. The motion data 

may be described with relative values of a local coordinate system of various 
joints, or may be described with absolute values of a world coordinate system of 
character data itself. Processing of moving the character by the motion data is 
carried out to be linked with the audio reproduction. 

3 5 [0020] 

FIG. 5 shows an example of images in a state displayed. In case of this 
example, a character such as a person created based on the character data is 
displayed on a display panel as shown in FIGS. 5(a), 5(b) and 5(c). In this 
example, with instructions on character's motion in motion data, as is shown in 

4 0 FIG 5(a), a state in which the character raises one hand is displayed at a certain 

position in music reproduction, when the music reproduction proceeds from that 
state, the display changes to a state in which one hand of the character falls and 
the other hand is raised as shown in FIG 5(b). When the music reproduction 
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further proceeds from that state, the display changes to a state in which both 
hands of the character are raised as shown in FIG. 5(c). 
[0021] 

The character data and motion data for such display have the structure 
5 shown in FIG. 6, for example. Specifically, for example, audio data (music data) 

101, character data 102, and motion data 103 constitute one audio data file. In 
this case, some flag, for example, is put to audio data to indicate that the character 
data and motion data are added. In addition, the character data 102 is data on 
the shape of each part of the displayed character. The motion data 103 indicates 
10 a coordinate position to be changed with respect to a part (here, an arm part, for 

example) at a specific position of the character at a specific time of audio 
reproduction. The motion data shown in FIG. 6 is expressed in 
three-dimensional graphic description language called VRML (virtual reality 
modeling language). 
15 [0022] 

FIG. 7 shows an example in which the audio data, character data and 
motion data are packed into a piece of data. 

[0023] 

With the character data and motion data having the above structure being 
2 0 added to audio data when reproducing the audio, such a character is displayed on 

the display panel that has motion corresponding to the audio reproduction. 
Being displayed in this manner, for example, if there is choreography 
corresponding to the reproduced music, the choreography will be indicated with 
the display of the character, and so the choreography can be studied by looking at 

2 5 the display. 

[0024] 

Since the character thus displayed is based on data added to audio data 
itself, it is possible to change the displayed character and motion in accordance 
with music to be reproduced, and the character suitable for each music piece can 

3 0 be displayed. Differently from the case of an image program for reproducing a 

movie and the like, because no image signal for reproducing a moving image is 
prepared, the data volume is very small, and therefore, such a recording medium 
for the audio data that has almost the same recording capacity as that of a 
recording medium for ordinary audio data can be used. Moreover, differently 

3 5 from the case of a typical image program, even after contents have been produced, 

data (character data, motion data) for displaying an image is easily modified and, 
for example, modification of a motion of a character and the like can be simply 
performed afterward. 
[0025] 

4 0 In addition, when the character data added to audio data is data for 

displaying a person and the like in three dimensional shape, by indicating a point 
of view (view angle) toward the displayed character with an operation of the key 
26 and the like in the reproduction apparatus, the character seen from an arbitrary 
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viewpoint can be displayed on the display panel 23. 
[BRIEF DESCRIPTION OF DRAWINGS] 
[0026] 

[FIG 1] A block diagram showing an example of a system configuration 
5 according to an embodiment of the present invention. 

[FIG 2] An explanatory diagram showing an example of a hierarchic 
structure for reproduction processing according to an embodiment of the present 
invention. 

[FIG 3] A flowchart showing an example of data processing according to 
10 an embodiment of the present invention. 

[FIG. 4] An explanatory diagram showing an example of a processing 
state according to an embodiment of the present invention. 

[FIG 5] An explanatory diagram showing an example of display 
according to an embodiment of the present invention. 
15 [FIG. 6] An explanatory diagram showing an example of data according 

to an embodiment of the present invention. 

[FIG 7] An explanatory diagram showing an example of data packing 
according to an embodiment of the present invention. 
[DESCRIPTION OF REFERENCE SYMBOLS] 
2 0 [0027] 

1 1... recording medium, 12. ..data reader, 13. ..data processor, 14. ..audio 
reproduction processor, 15... digital/analog converter, 16L, 16R... amplifier, 17L, 
17R... speaker, 21... character-data processor, 22. ..image processor, 23. ..display 
panel, 24. ..controller, 25. ..memory, 26. ..operating key 
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[0 7] 



Content-Type: multipart/mixed; 

boundary" =.Next ParU)00_001 1.01 BFA9E7.2EE28580 

=.Next Part.000_001 1.01BFA9E7.2EE28580 

Content-Type: application/ATRAC3 

=.Next Part_000_001 1.01 BFA9E7.2EE28580 

Content-Type: data/character 

=_Next Part.000.001 1.01BFA9E7.2EE28580- 

Content-Type: data/motion 

=_Next Part.000.001 1.01 BFA9E7.2EE28580— 



=_Next Part(wed_Apr_19.1 1 :42:48.2000.705) 



[NAME OF DOCUMENT] DRAWINGS 



[Ell] [Fig. 1] 

1 1 recording medium 

12 data reading 

13 data processing 

1 4 audio reproduction processing 

2 1 character-data processing 

22 image processing 

23 display panel 
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25 memory 
24 control 

26 key 

i/^^r J^^tffcffil example of system configuration 



[EI 2] [Fig. 2] 

2 music data reproduction apparatus 

3 character engine 

3 a construction analysis module 

10 3b performance module 

3c display module 

example of hierarchic structure for reproduction 

processing 

15 IMS] [Fig. 3] 

511 Input data 

5 12 Are there character data and motion data? 

5 1 3 Extract music data 

5 1 4 Reproduce music data 

2 0 S 1 5 Separate music data from other data 

516 Analyze construction 

517 Perform 

5 1 8 Display on screen 

l=f^y~ ^ 9 music data 

2 5 7 s ' — 9 ^SFlJ example of data processing 



[[34] [Fig. 4] 

2 music data reproduction apparatus 

3 character engine 

3 0 3a construction analysis module 

3b performance module 

3 c display module 

7 9 9 7* ' — 9 b ^ — v' B l/^r — 9 #f $k*f— 9 music data including 



character data and motion data 
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[BIS] [Fig. 5] 

^c?T^f 7* performed image 

[El 6] [Fig. 6] 

5 101 music data 

102 character data 

1 03 motion data 

"T* 1 — ^ example of data 

10 [m?] [Fig. 7] 

Isf^cX' — ^ music data 
^t7^^ x — ^ character data 
^ ~ >' a ^ motion data 

T^^&^&CDffi} example of data packing 

15 
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[NAME OF DOCUMENT] ABSTRACT 
[SUMMARY] 

[OBJECT] To enable an image attached to audio to be displayed comparatively 
with ease. 

5 [MEANS FOR SOLVING] When reproducing audio data to which character data 

for displaying a character having a specific shape is added, the character data is 
analyzed to generate data on an image having the shape specified by the character 
data and to display the image correspondingly to the reproduction of the audio 
data. In addition, when motion data for indicating motion of the character 
1 0 having the shape specified by the character data is added, the motion indicated by 

the motion data is displayed correspondingly to the reproduction of the audio data. 
Further, based on a predetermined input operation, the character having a 
three-dimensional shape seen from an arbitrary viewpoint is displayed. 
[SELECTED DRAWING] Figure 4 
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